This invention relates to an image processing method and apparatus for effecting a conversion from low-resolution information to high-resolution information in an image output apparatus such as a printer for outputting input image information upon enlarging the image information by a zoom function, or in communication between devices whose resolutions differ.
Various methods of converting input low-resolution information to high-resolution information have been proposed. In these proposed prior-art methods, the conversion processing differs depending upon type of image to be processed. Examples of such images are a multivalued image having gray-scale information for each pixel, a binary image binarized by pseudo-halftone processing such as in the dither method or error-diffusion method, a binary image or character image binarized based upon a fixed threshold value, etc. In the present invention, the image dealt with is a multivalued image such as a natural picture having gray-scale information for each pixel. However, the conventional interpolation method used is generally nearest neighbor interpolation, in which pixel values nearest to an interpolated point are arrayed, as shown in FIG. 1, or bi-linear interpolation in which, as shown in FIG. 2, a pixel value E is decided by the following operation based upon the distances of four points (the pixel values of which are assumed to be A, B, C, D) surrounding the interpolated point: EQU E=(1-i)(1-j)A+i.multidot.(1-j)B+(1-i)C+ijD
where, if the pixel-to-pixel distance is 1, the pixel value E is at a distance i from the A along the horizontal axis and a distance j from A along the vertical axis (i.ltoreq.1, j.ltoreq.1).
In image transmission such as in a color facsimile apparatus (FAX), it is essential that the image information be compressed before being transmitted over a transmission line. The JPEG (Joint Photographic Experts Group) method has been established in recent years as an international standard for encoding still color images. According to the JPEG method, image information is compressed by quantization of transformation coefficients by DCT (discrete cosine transform) and entropic encoding of transformation coefficients after quantization.
For the purpose of application to communication between devices having different resolutions, much research is now being carried out into techniques through which direct decompression from compressed information and conversion of resolution may be executed at the same time. Several methods in which the aforesaid DCT and conversion of resolution are combined have been proposed in Japanese Patent Application Laid-Open Nos. 4-229382, 4-333989; etc.). According to these proposed methods, first DCT is applied. When the resulting image is enlarged, "0" is inserted for new high-frequency components. When the resulting image is reduced, on the other hand, block size is changed by dropping the currently existing high-frequency components. This is followed by applying IDCT.
The transformation coefficients of two-dimensional DCT of (NXN) pixels are obtained by ##EQU1## The IDCT is obtained by ##EQU2## together with ##EQU3##
If we let F(u,v)! represent the matrix of NXN- number of F(u,v) and let F(u,v)!! represent a decompressed matrix in which "0" has been substituted for high-frequency components, then we have ##EQU4## (the DC component is at the upper left).
This method makes it possible to perform a resolution conversion of high picture quality at the same time as decompression.
However, the example of the prior art described above has certain drawbacks.
The method of FIG. 1 is advantageous in that the arrangement is a simple one. In addition, this method is suitable when the image dealt with is a geometrical image having many vertical and horizontal lines. However, in a case where the method is applied to a natural picture or the like, pixel values are decided for every block enlarged. As a consequence, blocks become visually conspicuous and picture quality declines.
The method of FIG. 2 is generally used when enlarging a natural picture. Though picture quality is such that the picture is averaged and smoothened, blurring occurs at edge portions and at portions requiring sharpness. Furthermore, in case of an image obtained by scanning a map or a natural picture containing characters, vital information is not transmitted to the receiving party owing to blurring caused by interpolation.
Further, the method of conversion of resolution (here only enlargement is considered) combined with compression by orthogonal transformation described above is an epoch-making technique in which it is possible to readily restore spatial information, which has undergone a conversion in resolution, from compressed information. However, this approach is not necessarily appropriate for all types of images.
The cause of a deterioration in picture quality when using this technique will now be described in detail.
To simplify the description, an example will be described in which "0" is substituted for high-frequency components of image information which has undergone DCT at a block size of 4.times.4, the information is converted to a block size of 8.times.8 and IDCT is applied.
Two-dimensional DCT corresponds to carrying out separation of components at a base image corresponding to the basic vector of a one-dimensional DCT, and a one-dimensional DCT is applied to the image independently in the horizontal and vertical direction. Accordingly, the description will be limited to the one-dimensional transform.
In FIG. 3, (a) illustrates basic vectors of a fourth-order DCT, and (b) illustrates basic vectors of sequence numbers 0 to 3 of an eighth-order DCT.
Both sequence numbers 0 indicate DC components and the other sequence numbers indicate AC components.
In a case where a fourth-order DCT is applied, the transformation coefficients outputted after application of the DCT represent power with respect to each basic vector shown in (a) of FIG. 3. In a conversion of resolution by making a substitution from fourth order to eighth order, the values of the transformation coefficients obtained by the fourth-order DCT are handled as though they were calculated as is by an eighth-order base. (DC components require correction by a proportional computation owing to a disparity in block size.)
FIG. 4 illustrates the disparity between the fourth-order and eighth-order DCT bases of sequence number 1 shown in FIG. 3. The dashes lines illustrate the fourth-order base and the solid lines the eighth-order base. (Since the number of pixels used in the bases differ between the fourth- and eighth-order DCTs, the fourth-order base is illustrated together with the eighth-order base.) The portions indicated by the arrows in FIG. 4 show the disparity between the fourth-order base and the eighth-order base.
FIG. 5 illustrates the disparity between the fourth-order and eighth-order DCT bases of sequence number 3 shown in FIG. 3. As in FIG. 4, the dashes lines illustrate the fourth-order base and the solid lines the eighth-order base. The disparity between them is indicated by the arrows. A comparison of FIGS. 4 and 5 shows that the larger the sequence number, the greater the disparity. In order to replace the base of the fourth-order DCT by the base of the eighth-order DCT, fine portions between adjacent pixels are expressed as a matter of course and portions of improved picture quality appear. Conversely, however, owing to the disparity between the bases of the fourth-order DCT and eighth-order DCT in each sequence, fine information is created arbitrarily where information is not present.
FIGS. 6A to 6D show an example in which an edge in an image has been subjected to processing according to the above-described example of the prior art. FIG. 6A illustrates one block of input image information in real space. In a case where this technique is actually applied to a device connected to a host computer of a printer or the like, steep edges often are generated in characters, line images and computer graphics CG).
FIG. 6B illustrates the results of subjecting the information shown in FIG. 6A to DCT at a block size of 4.times.4; FIG. 6C illustrates the results of substituting "0" into the high-frequency components on the basis of the transformation coefficients of FIG. 6A; and FIG. 6D illustrates the results of applying an 8.times.8 IDCT on the basis of the transformation coefficients of FIG. 6C. Ordinarily, with a compression method based upon DCT, an operation is applied in which transformation coefficients are quantized or amount of code is reduced by cutting high-frequency components. However, a detailed description of this is not given in this invention.
As will be appreciated from the results of IDCT shown in FIGS. 6A to 6D, ringing-type noise is produced also at portions other than edges owing to replacement of the fourth-order base by the eighth-order base. The reason for this is that from sequence numbers 0 to 3 of the eighth-order base, power can be substituted and filled by replacement of the base but power is "0" with respect to basic vectors in the region of higher frequency, namely from sequence numbers 4 to 7. The above-mentioned ringing noise is the result of this imbalance. In other words, irrespective of the fact that a portion corresponding to the above-described disparity between bases is added on by adopting the eighth-order base, this is not dealt with at all in the high-frequency region. Consequently, so-called "mosquito noise" is produced just as when high frequencies are cut at the time of quantization.